Configuring Manual and Scheduled Refresh in Power BI

Course: Microsoft Power BI Training

Topic: Configuring Manual and Scheduled Refresh

Course Link: Microsoft Power BI Training at JustAcademy

Demo Link: Register for a Course Demo

JustAcademy Website: Visit JustAcademy

1. Introduction to Data Refresh in Power BI

Data refresh is an essential feature of Microsoft Power BI that keeps reports and dashboards updated with the latest information from their connected data sources. Business data changes regularly as new sales transactions, customer records, inventory updates, financial entries, and operational activities are recorded. Refreshing data allows Power BI reports to reflect these changes.

When a report is created using imported data, Power BI stores a copy of the data in its semantic model. This imported data does not automatically change whenever the original database, spreadsheet, or other source is updated. A refresh process retrieves updated information from the source and updates the semantic model.

Power BI provides different ways to refresh data, including manual refresh and scheduled refresh. Manual refresh is initiated by a user, while scheduled refresh runs automatically according to a configured timetable.

1.1 Why Is Data Refresh Important?

• Maintains the accuracy and relevance of reports.
• Ensures business users can analyze updated information.
• Reduces the need to manually rebuild reports when source data changes.
• Supports daily, weekly, and periodic business reporting.
• Helps management make decisions using current business information.
• Improves reporting consistency by establishing a regular update process.

1.2 Example of Data Refresh

Suppose a company maintains its sales transactions in an Excel file. A Power BI report imports the file and calculates total sales of ₹5,00,000. Later, the sales team adds new transactions worth ₹50,000 to the Excel file.

Until the Power BI semantic model is refreshed, the report may continue displaying the previous total. After a successful refresh, the report can display ₹5,50,000, assuming the new transactions are included in the source data and the model calculations are correct.

2. Understanding Manual Refresh

Manual refresh is a process in which a user explicitly initiates a data refresh instead of waiting for an automatic schedule. It is useful when updated data must be reflected in a report immediately or when testing changes to a data source.

Power BI supports several refresh actions. It is important to understand the difference between refreshing data in Power BI Desktop and refreshing a published semantic model in the Power BI Service.

2.1 Manual Refresh in Power BI Desktop

Power BI Desktop is used to connect to data sources, transform data, build data models, create calculations, and design reports.

To manually refresh data in Power BI Desktop:

1. Open Power BI Desktop.
2. Open the required Power BI report file with the .pbix extension.
3. Confirm that the data source is accessible.
4. Go to the Home tab on the ribbon.
5. Click the Refresh button.
6. Wait while Power BI retrieves the latest data from the connected sources.
7. Review the refreshed tables, visualizations, and calculations.
8. Save the report after verifying the results.

Power BI executes the applicable queries and updates the imported data in the local semantic model. If the source has changed, the report visuals can reflect the updated values.

2.2 Manual Refresh in Power BI Service

Power BI Service is the cloud-based platform used to publish, share, manage, and monitor Power BI reports and semantic models.

To manually refresh a published semantic model:

1. Open Power BI Service and sign in with the appropriate account.
2. Navigate to the relevant workspace.
3. Locate the semantic model associated with the report.
4. Open the available actions menu for that semantic model.
5. Select the Refresh now option, where available.
6. Wait for the refresh operation to finish.
7. Check the refresh history to confirm whether the operation succeeded.
8. Open the associated report and verify the updated data.

The exact interface and available options can vary depending on the workspace experience, permissions, licensing, and semantic model configuration.

2.3 Manual Refresh in Power Query Editor

Power Query Editor is used to connect to data sources and perform transformations such as filtering rows, changing data types, removing duplicates, splitting columns, and merging tables.

During development, users may preview data in Power Query Editor to check whether a query returns the expected results. Refreshing a preview is not necessarily the same as refreshing the entire semantic model.

1. Open Power BI Desktop.
2. Select Home and then Transform data.
3. Select the required query from the Queries pane.
4. Use the preview refresh option when available.
5. Review the updated preview and verify the applied steps.
6. Select Close and Apply to apply query changes and load the resulting data into the model.

Important: Refreshing a Power Query preview may only update the preview. Use the appropriate model refresh process when the objective is to update the report's imported data.

3. Understanding Scheduled Refresh

Scheduled refresh allows Power BI to refresh a supported semantic model automatically at predefined times. Once configured, it reduces the need for users to initiate every refresh manually.

Scheduled refresh is particularly useful for organizations that receive updated source data at regular intervals and need reports to remain current throughout the working day or reporting cycle.

3.1 Benefits of Scheduled Refresh

• Automation: Data refreshes without requiring a user to initiate each operation.
• Consistency: Reports can be updated according to a repeatable schedule.
• Time savings: Reduces repetitive manual work.
• Better reporting workflows: Helps align refreshed data with business reporting deadlines.
• Improved availability: Users can access recently refreshed information when the refresh completes successfully.
• Monitoring: Refresh history can help administrators identify failures and recurring problems.

3.2 Examples of Scheduled Refresh

These are illustrative schedules. The appropriate frequency depends on the data source, business requirements, licensing, capacity, and Power BI service limits.

4. Prerequisites for Configuring Scheduled Refresh

Before configuring scheduled refresh, verify that the semantic model can access its source data and that the required settings are available.

4.1 Data Source Connectivity

Power BI must be able to connect to the source. For example, an Excel workbook stored in OneDrive or SharePoint may use cloud authentication, while an on-premises SQL Server database may require an appropriately configured data gateway.

4.2 Authentication and Credentials

Credentials must be valid and have sufficient permissions to read the required data. Expired passwords, revoked permissions, changed authentication settings, and expired tokens can cause refresh failures.

4.3 Data Gateway

A data gateway provides a secure connection between Power BI Service and supported data sources located within a private network or on-premises environment.

A gateway may be required when Power BI Service cannot directly access the source. It must be installed or configured appropriately, remain available, and have valid data source credentials and permissions.

4.4 Workspace and Semantic Model Permissions

The user configuring refresh must have sufficient permissions to manage the semantic model and its settings. Some configuration options may depend on the user's role, workspace setup, license, or capacity.

4.5 Refresh Limits and Capacity

Scheduled refresh frequency and other operational limits depend on the applicable Power BI licensing and capacity. Before setting up a schedule, review current service limits and ensure that the planned refresh frequency is supported.

5. Step-by-Step: Configuring Scheduled Refresh

The following procedure describes the general process for configuring scheduled refresh for a published semantic model in Power BI Service.

Step 1: Publish the Power BI Report

1. Open the report in Power BI Desktop.
2. Verify the data connections, transformations, relationships, and measures.
3. Save the .pbix file.
4. Click Publish on the Home ribbon.
5. Select the destination workspace.
6. Wait for the publishing process to complete.

Publishing transfers the report and its associated semantic model to the Power BI Service. Publishing alone does not guarantee that future source data changes will be refreshed automatically.

Step 2: Open the Workspace

1. Open Power BI Service at app.powerbi.com.
2. Sign in with the account that has access to the workspace.
3. Open the workspace where the report was published.
4. Locate the semantic model associated with the report.

Step 3: Open Semantic Model Settings

1. Locate the semantic model in the workspace.
2. Open its context menu or settings page.
3. Review the available data source, gateway, and refresh settings.
4. Confirm that the required configuration options are available to your account.

Step 4: Configure Data Source Credentials

1. Locate the credentials or data source connection section.
2. Select the appropriate authentication method.
3. Sign in or provide the required credentials.
4. Save the configuration.
5. Resolve any authentication errors before continuing.

The exact authentication method depends on the source, which may include organizational accounts, database credentials, OAuth-based authentication, or other supported methods.

Step 5: Configure Gateway Settings if Required

1. Determine whether the source requires an on-premises data gateway.
2. Ensure that the appropriate gateway is installed and running.
3. Confirm that the semantic model is mapped to the correct gateway data source when required.
4. Verify that the gateway connection and credentials are valid.
5. Check that the gateway administrator has configured the necessary permissions.

If the source is fully accessible through a supported cloud connection, a gateway may not be necessary.

Step 6: Enable Scheduled Refresh

1. Open the semantic model's settings.
2. Locate the scheduled refresh section.
3. Enable the scheduled refresh option if it is available.
4. Select the appropriate refresh frequency, such as daily or weekly, where supported.
5. Choose the required refresh time slots.
6. Confirm the applicable time zone and schedule.
7. Save or apply the configuration.

The settings available may differ by source, semantic model type, and Power BI environment. If scheduled refresh is unavailable, investigate the model configuration and applicable service limitations.

Step 7: Test the Refresh Configuration

1. Use Refresh now to perform an initial test, if available.
2. Wait for the refresh operation to complete.
3. Open the refresh history.
4. Verify that the operation succeeded.
5. Open the report and check important measures and visuals.
6. Confirm that the refreshed data matches the expected source data.

Testing the configuration helps identify connection and authentication issues before relying on the scheduled refresh.

6. Configuring Refresh Frequency and Timing

Refresh frequency determines how often Power BI attempts to retrieve updated data. A suitable schedule balances freshness, source system performance, service limits, and business requirements.

6.1 Choosing a Suitable Frequency

6.2 Time Zone Considerations

Refresh schedules must be interpreted using the applicable time zone and service configuration. A schedule configured in one time zone may not correspond to the expected local reporting time if the time zone is misunderstood.

For example, if a sales report must be available by 9:00 AM local time, the refresh should be scheduled with enough time for the operation to finish before users begin reviewing the report.

6.3 Refresh Duration

A scheduled refresh can take longer when the dataset contains large volumes of data, complex transformations, slow database queries, or many connected sources. Avoid scheduling several demanding operations at the same time if doing so overloads the source system or shared capacity.

7. Manual Refresh vs. Scheduled Refresh

Key point: Manual and scheduled refresh are complementary approaches. Manual refresh is useful for immediate actions, while scheduled refresh is useful for recurring updates.

8. Understanding Import Mode and DirectQuery

Refresh behavior depends partly on how Power BI connects to its data sources.

8.1 Import Mode

In Import mode, Power BI loads a copy of source data into its semantic model. Reports query the imported data, so source changes generally need to be brought into the model through refresh.

• Provides fast interactive analysis for many workloads.
• Supports a wide range of data modeling and calculation capabilities.
• Requires refresh to incorporate subsequent source changes.
• May consume memory and storage depending on model size.

8.2 DirectQuery Mode

In DirectQuery mode, Power BI queries the underlying source for relevant data rather than importing the entire source dataset into the semantic model in the same way as Import mode.

• Can provide access to more current source data, subject to caching and query behavior.
• Depends on source system availability and performance.
• May require refresh for certain model metadata or other supported elements, even though source data queries work differently.
• Does not use the same full imported-data refresh pattern as a typical Import model.

8.3 Choosing Between Import and DirectQuery

9. Incremental Refresh

Incremental refresh is a feature designed to refresh only selected portions of data rather than repeatedly processing the entire historical dataset. It is especially useful for large tables that contain records across many dates or periods.

9.1 Why Use Incremental Refresh?

• Reduces the amount of historical data processed during routine refreshes.
• Can improve refresh performance for large datasets.
• May reduce the time and resources needed for repeated refresh operations.
• Allows older historical partitions to be retained while recent data is refreshed according to policy.

9.2 Basic Configuration Process

1. Open the required table in Power Query Editor.
2. Identify a suitable date or date-time column.
3. Create and configure the required RangeStart and RangeEnd parameters.
4. Filter the table using the appropriate date range.
5. Confirm that the filter folds to the source where supported and required for the intended design.
6. Configure the incremental refresh policy for the table.
7. Apply the changes and publish the model.
8. Validate the policy and refresh behavior in the Power BI Service.

Availability and behavior depend on the semantic model type, source connector, licensing, and capacity. Incremental refresh is not simply a switch that guarantees every refresh processes only a small amount of data; the policy and model must be configured correctly.

10. Monitoring Refresh History

Refresh history helps users and administrators determine whether refresh operations completed successfully, failed, or encountered errors.

10.1 How to Check Refresh History

1. Open Power BI Service.
2. Navigate to the workspace containing the semantic model.
3. Open the semantic model's settings or available actions.
4. Locate the refresh history section.
5. Review the recent refresh attempts.
6. Check the status, start time, completion time, and any available error details.
7. Investigate recurring failures and correct the underlying problem.

10.2 Common Refresh Statuses

Exact status labels and available diagnostic details may vary across the Power BI interface and refresh type.

11. Common Refresh Problems and Troubleshooting

Manual and scheduled refresh can fail for several reasons. A structured troubleshooting approach helps identify whether the issue originates in the source, credentials, gateway, query, model, or service configuration.

11.1 Invalid Credentials

Problem: Power BI cannot authenticate with the data source.

Possible causes:

• A password has changed or expired.
• An authentication token has expired.
• The account has lost permission to access the source.
• The selected authentication method is incorrect.

Solution: Open the semantic model settings, update the data source credentials, confirm access permissions, and run a test refresh.

11.2 Gateway Connection Failure

Problem: A semantic model that depends on an on-premises source cannot refresh.

Possible causes:

• The gateway machine is switched off.
• The gateway service is stopped.
• The gateway is not mapped to the correct data source.
• The gateway credentials or permissions are invalid.
• Network connectivity or firewall rules prevent the connection.

Solution: Confirm gateway availability, verify the source configuration, review credentials and network access, and retry the refresh.

11.3 Data Source Unavailable

Problem: Power BI cannot access a file, database, API, or other connected source.

Solution: Verify the source location, network connection, server availability, file permissions, and any relevant connection settings.

11.4 Query Errors

Problem: A Power Query transformation fails during refresh.

Possible causes:

• A column was renamed or removed from the source.
• The source contains unexpected values.
• A data type conversion fails.
• A query references an unavailable table or field.
• The structure of an Excel file or API response has changed.

Solution: Open Power Query Editor in Power BI Desktop, inspect the Applied Steps, identify the failing transformation, correct the query, test it, and republish the updated model when necessary.

11.5 Refresh Timeout

Problem: A refresh operation takes too long or exceeds a supported execution limit.

Possible causes:

• Large data volumes.
• Slow source queries.
• Complex transformations.
• Network delays.
• High source-system or capacity workload.

Solution: Optimize source queries, remove unnecessary columns and rows, improve query folding where possible, consider incremental refresh, and schedule demanding operations appropriately.

11.6 Scheduled Refresh Does Not Run as Expected

Problem: The semantic model does not update at the expected time.

Solution:

1. Confirm that scheduled refresh is enabled.
2. Check the configured frequency, time slots, and time zone.
3. Review the refresh history for failures.
4. Verify credentials and gateway availability.
5. Check applicable refresh limits and capacity conditions.
6. Confirm that the source data was updated before the scheduled refresh began.

11.7 Refresh Succeeds but Report Values Appear Incorrect

Problem: The refresh status indicates success, but the report does not display the expected results.

Possible causes:

• The source data itself has not been updated.
• The query filters exclude the new records.
• The report uses a different semantic model than expected.
• Relationships or measures affect the displayed totals.
• Some data sources or tables follow a different update process.

Solution: Compare source data with the refreshed model, review query filters and relationships, verify the report's semantic model, and test the relevant measures and visual filters.

12. Best Practices for Manual and Scheduled Refresh

• Choose a refresh schedule that matches actual business requirements.
• Verify data source credentials before deploying a production report.
• Configure and monitor gateways for sources that require them.
• Check refresh history regularly, particularly for critical business reports.
• Avoid unnecessary refresh operations that increase source workload.
• Optimize Power Query transformations and remove unused columns and rows.
• Use incremental refresh when appropriate for large, date-based datasets.
• Allow enough time for refresh completion before scheduled reporting meetings.
• Document refresh frequency, source ownership, and troubleshooting procedures.
• Test important reports after significant source schema changes.
• Review service limits and licensing requirements before increasing refresh frequency.
• Do not assume that publishing a report automatically enables scheduled refresh.
• Do not assume that every connection type uses the same refresh behavior.
• Use appropriate access controls to protect credentials and sensitive data.

13. Practical Example: Sales Dashboard Refresh

Consider a retail organization that uses Power BI to monitor daily sales. The sales team stores transactions in a SQL Server database, and management reviews a dashboard every morning.

13.1 Business Requirements

• Display the latest available sales transactions.
• Update total revenue, order counts, and product performance.
• Make the dashboard available before the morning meeting.
• Provide a way to refresh data manually when urgent updates are required.

13.2 Implementation Steps

1. Connect Power BI Desktop to the SQL Server database.
2. Use Power Query to clean the sales data and assign appropriate data types.
3. Create relationships between sales, product, customer, and date tables where required.
4. Build measures and report visuals for revenue and order analysis.
5. Publish the report to the appropriate Power BI workspace.
6. Configure the required source credentials and gateway settings.
7. Schedule refresh before the management meeting, allowing sufficient time for completion.
8. Test the configuration with a manual refresh.
9. Monitor refresh history and verify key sales figures.

13.3 Example DAX Measures

Total Sales = SUM(Sales[SalesAmount])

Total Orders = DISTINCTCOUNT(Sales[OrderID])

Average Order Value =
DIVIDE([Total Sales], [Total Orders], 0)

These measures calculate total sales, distinct orders, and average order value based on the model data. Refresh updates the underlying imported data when applicable; the measures then evaluate against the current model data when the report is queried.

13.4 Expected Outcome

After a successful refresh, the dashboard can reflect newly available transactions, provided that the source data, queries, relationships, and report filters include those transactions. Management can use the refreshed report for more informed sales analysis.

14. Practical Exercise

Objective: Practice configuring manual and scheduled refresh using a sample Power BI sales report.

Exercise Tasks

1. Create an Excel file containing sales records with Order ID, Order Date, Product, Quantity, Unit Price, and Sales Amount columns.
2. Import the Excel file into Power BI Desktop.
3. Transform the data using Power Query Editor.
4. Create a total sales measure.
5. Build a report showing total sales by product and date.
6. Refresh the data manually in Power BI Desktop.
7. Add a new transaction to the source Excel file and test the refresh process.
8. Publish the report to Power BI Service.
9. Review the semantic model settings and determine which credentials or connection settings are needed.
10. Configure scheduled refresh if supported by the source, model, and account.
11. Run a manual refresh in the service and check the refresh history.
12. Record any errors and document how they were resolved.

Exercise Questions

1. What happens when the source file changes but the imported model is not refreshed?
2. What is the difference between refreshing a Power Query preview and refreshing a semantic model?
3. Why might a gateway be necessary?
4. How can you verify that a scheduled refresh succeeded?
5. What steps would you take if the refresh fails because of invalid credentials?

15. Interview Questions and Answers

Q1. What is data refresh in Power BI?

Answer: Data refresh is the process of updating a Power BI semantic model with data from its connected sources or performing the relevant refresh operations supported by the model's connection mode.

Q2. What is manual refresh?

Answer: Manual refresh is a refresh operation initiated by a user, for example, by clicking Refresh in Power BI Desktop or Refresh now in Power BI Service.

Q3. What is scheduled refresh?

Answer: Scheduled refresh automatically initiates supported refresh operations at configured times in Power BI Service.

Q4. What is the main difference between manual and scheduled refresh?

Answer: Manual refresh requires a user to initiate the operation, whereas scheduled refresh runs according to a configured schedule.

Q5. Does publishing a Power BI report automatically configure scheduled refresh?

Answer: No. Scheduled refresh may require additional configuration, including source credentials, gateway settings, refresh frequency, and other applicable settings.

Q6. What is a data gateway?

Answer: A data gateway enables Power BI Service to securely access supported data sources that are not directly accessible from the cloud, such as certain on-premises databases.

Q7. Why can scheduled refresh fail?

Answer: Common causes include invalid credentials, unavailable sources, gateway failures, query errors, network problems, timeouts, and service or capacity limitations.

Q8. How do you check whether a refresh succeeded?

Answer: Review the semantic model's refresh history in Power BI Service and verify that the operation completed successfully.

Q9. What is incremental refresh?

Answer: Incremental refresh is a configuration that allows supported semantic models to refresh selected data partitions or periods rather than repeatedly processing all historical data.

Q10. Is scheduled refresh always required for DirectQuery?

Answer: DirectQuery retrieves source data through queries rather than following the same imported-data refresh pattern. Some model and metadata operations may still require refresh, depending on the configuration.

Q11. How can refresh performance be improved?

Answer: Performance can often be improved by optimizing source queries, removing unnecessary data, improving query folding, using incremental refresh where suitable, and avoiding unnecessary source workload.

Q12. What should you do if a refresh succeeds but the report still shows unexpected values?

Answer: Verify the source data, query filters, semantic model, relationships, measures, and report filters to identify why the expected records or calculations are not appearing.

16. Quick Revision Checklist

• Understand the purpose of data refresh in Power BI.
• Know how to refresh data manually in Power BI Desktop.
• Know how to initiate a manual refresh in Power BI Service.
• Understand the difference between query preview refresh and model refresh.
• Know how to locate semantic model settings.
• Understand scheduled refresh frequency and timing.
• Verify credentials and gateway configuration where applicable.
• Understand the differences between Import and DirectQuery modes.
• Know the purpose of incremental refresh.
• Check refresh history to identify failed operations.
• Understand common causes of refresh failures.
• Follow best practices for reliability, performance, and data security.

17. Conclusion

Configuring manual and scheduled refresh is an important part of maintaining reliable Power BI reports. Manual refresh allows users to initiate updates when required, while scheduled refresh automates supported refresh operations according to a defined timetable.

Successful refresh configuration depends on data source connectivity, valid credentials, gateway availability when required, appropriate scheduling, and regular monitoring. Understanding refresh history and common troubleshooting methods helps reduce reporting interruptions and improves confidence in business analytics.

By practicing these techniques, learners can manage Power BI data refresh workflows more effectively and maintain reports that reflect the latest successfully processed data.

18. Additional Learning Resources

• Explore Microsoft Power BI Training at JustAcademy
• Register for a Power BI Course Demo
• Visit the JustAcademy Website
• Microsoft Learn: Data Refresh in Power BI
• Microsoft Learn: Configure Scheduled Refresh
• Microsoft Learn: Incremental Refresh Overview
• Microsoft Learn: On-Premises Data Gateway